The invention relates to digital electronic circuitry. More particularly, the invention relates to a method and apparatus to determine when all outstanding fetches, i.e., memory read requests, are complete in a computer input/output system.
An input/output (xe2x80x9cI/Oxe2x80x9d) system can serve several I/O devices. It may be desirable to replace one of the devices during the operation of the I/O system while not affecting the I/O traffic associated with the other devices, a procedure known as on-line replacement (xe2x80x9cOLRxe2x80x9d). Also one device may need to be reset due to an error associated with that device, and again it is desirable to not affect the traffic of the devices. An I/O system may have multiple outstanding memory read requests on behalf of the device that requires resetting or replacing. These outstanding memory requests may result in non-coherent data being stored in the I/O system for the device. It is possible that this non-coherent data may be returned to the device after it is replaced or reset. This could result in stale data being given to the device. Before removing the data that is stored in the I/O system for the device it is necessary to know that all outstanding memory requests have been satisfied. In large systems this can theoretically take a long time, even though the majority of the requests are completed quickly. Two examples of why a request may take a long time to complete:
(1) The system is extremely busy; or
(2) Part of the system has failed and the request must wait for a timeout.
It is undesirable to have to wait the maximum time the request would take to complete (either normal completion or a timeout) each time a I/O device needs to be reset or replaced. Examples of times when it is desirable to know that all fetches for a device (or all the devices) have been completed include:
(1) OLR of one or multiple devices;
(2) OLR of I/O system, in which case the system needs to know ALL fetches are complete in order to shut down the I/O system in a systematic way; and
(3) To abort a direct memory access (xe2x80x9cDMAxe2x80x9d) sequence for a device due to some abnormal behavior (or error condition).
Prior solutions to the above problems have mainly been solved in operating system (xe2x80x9cOSxe2x80x9d) software. The OS would shut down the I/O devices that needed to be replaced or reset. Then, in order to make sure all the fetches for those devices were complete, a wait loop would be invoked. This wait loop would need to be as long as the maximum of the timeout for the fetches or theoretical limit of the memory return for the request. This wait loop could take a long time, in certain cases about 100 milliseconds. There are certain error recovery situations in which this time is longer than desirable as the I/O device would be unavailable during the entire wait loop time. In the vast majority of cases, the outstanding memory requests receive the required response relatively quickly, less than 2 microseconds. This means that the loop time is usually about 50,000 times longer than necessary.
In one respect, the invention is a method for determining completion of all of one or more operations for a particular input/output device initiated prior to an inquiry start time. The method provides one or more in-progress bits and an equal number of snapshot bits. Each in-progress bit corresponds to a respective associated operation for the input/output device. Each of the one or more in-progress bits is set when the associated operation is initiated. Each of the one or more in-progress bits is cleared when the associated operation is completed. The method copies, at the inquiry start time, all of the in-progress bits to the corresponding snapshot bits and clears a snapshot bit when the associated operations is completed. Finally, the method determines whether every one of said one or more snapshot bits is cleared.
In another respect, the invention is an apparatus for determining completion of all of one or more operations for a particular input/output device initiated prior to an inquiry start time. The apparatus comprises an input/output operator, one or more operation requesters, one or more in-progress bits and an equal number of snapshot bits. Preferably, the in-progress bits are grouped in a register, as are the snapshot bits. Each in-progress bit corresponds to a respective associated operation for the input/output devices, is set when the associated operation is initiated, and is cleared when the associated operations is completed.
In yet another respect, the invention is a computer readable medium on which is embedded a program. The embedded program comprises components that execute the above method or that correspond to the above apparatus components.
In comparison to the prior art, certain embodiments of the invention are capable of dramatically reducing waiting time. As an example, after the OS software has shut down the I/O devices that need to be replaced or reset, software can send the hardware a command indicating that it wants to know when all current outstanding memory fetches are complete. Instead of waiting the maximum time necessary for completion of a software loop, the system software can instead set and read a register that will indicate when memory fetches are complete. System software can loop on this register until all the fetches are complete. This reduces the amount of time that system software will have to wait to less than 2 microseconds in all but a very few cases.